Functional safety in autonomous driving

ABSTRACT

Autonomous driving of a vehicle in which computerized perception by the vehicle, including of its environment and of itself (e.g., its egomotion), is used to autonomously drive the vehicle and, additionally, can also be used to provide feedback to enhance performance, safety, and/or other attributes of autonomous driving of the vehicle (e.g., when certain conditions affecting the vehicle are determined to exist by detecting patterns in or otherwise analyzing what is perceived by the vehicle), such as by adjusting autonomous driving of the vehicle, conveying messages regarding the vehicle, and/or performing other actions concerning the vehicle.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority from U.S. Provisional PatentApplication 62/903,845 filed on Sep. 22, 2019 and incorporated byreference herein.

FIELD

This disclosure relates to vehicles (e.g., automobiles, trucks, buses,and other road vehicles) with an autonomous driving (a.k.a.,self-driving) capability.

BACKGROUND

Vehicles capable of autonomous driving (i.e., self-driving), which aredrivable without human control (e.g., by steering, accelerating, and/ordecelerating themselves autonomously) during at least part of their use,are becoming more prevalent.

For example, automobiles, trucks, and other road vehicles may becharacterized by various level of driving automation (e.g., any one oflevels 2 to 5 of SAE J3016 levels of driving automation), from partialdriving automation using one or more advanced driver-assistance systems(ADAS) to full driving automation.

Computerized perception by these vehicles of their environment and ofthemselves (e.g., their egomotion), based on various sensors (e.g.,cameras, lidar (light detection and ranging) devices, radar devices, GPSor other location sensors, inertial measurement units (IMUs), etc.), isused to autonomously drive them, by determining where and how to safelymove them and controlling actuators (e.g., of their powertrain, steeringsystem, etc.) to move them accordingly.

While it has greatly advanced, the computerized perception by thesevehicles may remain underutilized in some cases, and this may lead tosuboptimal performance, safety, and/or other attributes of autonomousdriving of these vehicles.

For these and other reasons, there is a need for improvements directedto vehicles with an autonomous driving capability.

SUMMARY

According to various aspects, this disclosure relates to autonomousdriving or various levels of driving assistance of a vehicle in whichcomputerized perception by the vehicle, including of its environment andof itself (e.g., its egomotion), is used to autonomously drive thevehicle and, additionally, can also be used to provide feedback toenhance performance, safety, and/or other attributes of autonomousdriving of the vehicle (e.g., when certain conditions affecting thevehicle are determined to exist by detecting patterns in or otherwiseanalyzing what is perceived by the vehicle), such as by adjustingautonomous driving of the vehicle, conveying messages regarding thevehicle, and/or performing other actions concerning the vehicle.

For example, according to one aspect, this disclosure relates to asystem for autonomous driving or various levels of driving assistance ofa vehicle. The system comprises an interface configured to receive datafrom sensors of the vehicle that include a camera and a lidar sensor,among others. The system also comprises a processing entity comprisingat least one processor and configured to: provide perception informationregarding perception of an environment of the vehicle and a state of thevehicle based on the data from the sensors, the perception informationcomprising a 3D model of the environment of the vehicle and informationabout a position of the vehicle; generate control signals forautonomously driving the vehicle based on the 3D model of theenvironment of the vehicle and the information about the position of thevehicle; and process the perception information, other than forgenerating the control signals for autonomously driving the vehiclebased on the 3D model of the environment of the vehicle and theinformation about the position of the vehicle, to determine whether apredefined condition affecting the vehicle exists and, if so, perform anaction concerning the vehicle based on the predefined condition.

According to another aspect, this disclosure relates to a system forautonomous driving or various levels of driving assistance of a vehicle.The system comprises an interface configured to receive data fromsensors of the vehicle that include a camera and a lidar sensor, amongothers. The system also comprises a processing entity comprising atleast one processor and configured to: provide perception informationregarding perception of an environment of the vehicle and a state of thevehicle based on the data from the sensors, the perception informationcomprising a 3D model of the environment of the vehicle and informationabout a position of the vehicle; generate control signals forautonomously driving the vehicle based on the 3D model of theenvironment of the vehicle and the information about the position of thevehicle; and process the perception information to detect a pattern inthe perception information indicative of a predefined conditionaffecting the vehicle.

According to another aspect, this disclosure relates to non-transitorycomputer-readable media comprising instructions executable by aprocessing apparatus for autonomous driving or various levels of drivingassistance of a vehicle, wherein the instructions, when executed by theprocessing apparatus, cause the processing apparatus to: receive datafrom sensors of the vehicle that include a camera and a lidar sensor,among others; provide perception information regarding perception of anenvironment of the vehicle and a state of the vehicle based on the datafrom the sensors, the perception information comprising a 3D model ofthe environment of the vehicle and information about a position of thevehicle; generate control signals for autonomously driving the vehiclebased on the 3D model of the environment of the vehicle and theinformation about the position of the vehicle; and process theperception information, other than for generating the control signalsfor autonomously driving the vehicle based on the 3D model of theenvironment of the vehicle and the information about the position of thevehicle, to determine whether a predefined condition affecting thevehicle exists and, if so, perform an action concerning the vehiclebased on the predefined condition.

According to another aspect, this disclosure relates to non-transitorycomputer-readable media comprising instructions executable by aprocessing apparatus for autonomous driving or various levels of drivingassistance of a vehicle, wherein the instructions, when executed by theprocessing apparatus, cause the processing apparatus to: receive datafrom sensors of the vehicle that include a camera and a lidar sensor,among others; provide perception information regarding perception of anenvironment of the vehicle and a state of the vehicle based on the datafrom the sensors, the perception information comprising a 3D model ofthe environment of the vehicle and information about a position of thevehicle; generate control signals for autonomously driving the vehiclebased on the 3D model of the environment of the vehicle and theinformation about the position of the vehicle; and process theperception information to detect a pattern in the perception informationindicative of a predefined condition affecting the vehicle.

According to another aspect, this disclosure relates to a method forautonomous driving or various levels of driving assistance of a vehicle.The method comprises: receiving data from sensors of the vehicle thatinclude a camera and a lidar sensor, among others; providing perceptioninformation regarding perception of an environment of the vehicle and astate of the vehicle based on the data from the sensors, the perceptioninformation comprising a 3D model of the environment of the vehicle andinformation about a position of the vehicle; generating control signalsfor autonomously driving the vehicle based on the 3D model of theenvironment of the vehicle and the information about the position of thevehicle; and processing the perception information, other than forgenerating the control signals for autonomously driving the vehiclebased on the 3D model of the environment of the vehicle and theinformation about the position of the vehicle, to determine whether apredefined condition affecting the vehicle exists and, if so, perform anaction concerning the vehicle based on the predefined condition.

According to another aspect, this disclosure relates to a method forautonomous driving or various levels of driving assistance of a vehicle.The method comprises: receiving data from sensors of the vehicle thatinclude a camera and a lidar sensor, among others; providing perceptioninformation regarding perception of an environment of the vehicle and astate of the vehicle based on the data from the sensors, the perceptioninformation comprising a 3D model of the environment of the vehicle andinformation about a position of the vehicle; generating control signalsfor autonomously driving the vehicle based on the 3D model of theenvironment of the vehicle and the information about the position of thevehicle; and processing the perception information to detect a patternin the perception information indicative of a predefined conditionaffecting the vehicle.

These and other aspects of this disclosure will now become apparent tothose of ordinary skill upon review of a description of embodiments inconjunction with accompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

A detailed description of embodiments is provided below, by way ofexample only, with reference to accompanying drawings, in which:

FIG. 1 shows an embodiment of a vehicle capable of autonomous driving;

FIG. 2 shows an example of a scene of an environment of the vehicle;

FIG. 3 shows examples of components of the vehicle;

FIG. 4 shows an embodiment of a control system of the vehicle;

FIGS. 5 to 7 shows an embodiment of a controller of the control systemof the vehicle;

FIG. 8 shows an example of a process implemented by the controller; and

FIGS. 9 and 10 show variants for the controller in other embodiments.

It is to be expressly understood that the description and drawings areonly for purposes of illustrating some embodiments and are an aid forunderstanding. They are not intended to and should not be limiting.

DETAILED DESCRIPTION OF EMBODIMENTS

FIGS. 1 to 5 show an embodiment of a vehicle 10 capable of autonomousdriving (i.e., self-driving) in an environment 11 of the vehicle 10. Inthis embodiment, the vehicle 10 is a road vehicle and its environment 11includes a road 19. The vehicle 10 is designed to legally carry peopleand/or cargo on the road 19, which is part of a public roadinfrastructure (e.g., public streets, highways, etc.). In this example,the vehicle 10 is an automobile (e.g., a passenger car).

The vehicle 10 is capable of autonomous driving in that, for at leastpart of its use, it is drivable without direct human control, includingby steering, accelerating, and/or decelerating (e.g., braking) itselfautonomously, to travel towards a destination. Although it can driveitself, in some embodiments, the vehicle 10 may be controlled orsupervised by a human driver in some situations. The vehicle 10 can thusbe characterized by any level of driving automation or assistance (e.g.,any one of levels 2 to 5 of SAE J3016 levels of driving automation),from partial driving automation using one or more advanceddriver-assistance systems (ADAS) to full driving automation.

As further discussed below, in this embodiment, computerized perceptionby the vehicle 10, including of its environment 11 and of itself (e.g.,its egomotion), is used to autonomously drive the vehicle 10 and,additionally, can also be used to provide feedback to enhanceperformance, safety, and/or other attributes of autonomous driving ofthe vehicle 10 (e.g., when certain conditions affecting the vehicle 10are determined to exist by detecting patterns in or otherwise analyzingwhat is perceived by the vehicle 10), such as by adjusting autonomousdriving of the vehicle 10, conveying messages regarding the vehicle 10,and/or performing other actions concerning the vehicle 10.

In this embodiment, the vehicle 10 comprises a frame 12, a powertrain14, a steering system 16, a suspension 18, wheels 20, a cabin 22, and acontrol system 15 that is configured to operate the vehicle 10autonomously (i.e., without human control) at least for part of its use.

The powertrain 14 is configured to generate power for the vehicle 10,including motive power for the wheels 20 to propel the vehicle 10 on theroad 19. To that end, the powertrain 14 comprises a power source (e.g.,a prime mover) that includes one or more motors. For example, in someembodiments, the power source of the powertrain 14 may comprise aninternal combustion engine, an electric motor (e.g., powered by abattery), or a combination of different types of motor (e.g., aninternal combustion engine and an electric motor). The powertrain 14 cantransmit power from the power source to one or more of the wheels 20 inany suitable way (e.g., via a transmission, a differential, a shaftengaging (i.e., directly connecting) a motor and a given one of thewheels 20, etc.).

The steering system 16 is configured to steer the vehicle 10 on the road19. In this embodiment, the steering system 16 is configured to turnfront ones of the wheels 20 to change their orientation relative to theframe 12 of the vehicle 10 in order to cause the vehicle 10 to move in adesired direction.

The suspension 18 is connected between the frame 12 and the wheels 20 toallow relative motion between the frame 12 and the wheels 20 as thevehicle 10 travels on the road 19. For example, the suspension 18 mayenhance handling of the vehicle 10 on the road 19 by absorbing shocksand helping to maintain traction between the wheels 20 and the road 19.The suspension 18 may comprise one or more springs, dampers, and/orother resilient devices.

The cabin 22 is configured to be occupied by one or more occupants ofthe vehicle 10. In this embodiment, the cabin 22 comprises a userinterface 70 configured to interact with one or more occupants of thevehicle and comprising an input portion that includes one or more inputdevices (e.g., a set of buttons, levers, dials, etc., a touchscreen, amicrophone, etc.) allowing an occupant of the vehicle 10 to inputcommands and/or other information into the vehicle 10 and an outputportion that includes one or more output devices (e.g., a display, aspeaker, etc.) to provide information to an occupant of the vehicle 10.The output portion of the user interface 70 may comprise an instrumentpanel (e.g., a dashboard) which provides indicators (e.g., a speedometerindicator, a tachometer indicator, etc.) related to operation of thevehicle 10.

The control system 15 is configured to operate the vehicle 10, includingto steer, accelerate, and/or decelerate (e.g., brake) the autonomousvehicle 10, autonomously (i.e., without human control) as the vehicle 10progresses towards a destination along a route on the road 19. Moreparticularly, the control system 15 comprises a controller 80 and asensing apparatus 82 to perform actions controlling the vehicle 10(e.g., actions to steer, accelerate, decelerate, etc.) to move ittowards its destination on the road 19, notably based on a computerizedperception of the environment 11 of the vehicle 10 and of the vehicle 10itself (e.g., its egomotion).

While its control system 15 enables it to drive itself, the vehicle 10may be controlled by a human driver, such as an occupant in the cabin22, in some situations. For example, in some embodiments, the controlsystem 15 may allow the vehicle 10 to be selectively operable eitherautonomously (i.e., without human control) or under human control (i.e.,by a human driver) in various situations (e.g., the vehicle 10 may beoperable in either of an autonomous operational mode and ahuman-controlled operational mode). For instance, in this embodiment,the user interface 70 of the cabin 22 may comprise an accelerator (e.g.,an acceleration pedal), a braking device (e.g., a brake pedal), and asteering device (e.g., a steering wheel) that can be operated by a humandriver in the cabin 22 to control the vehicle 10 on the road 19.

The controller 80 is a processing apparatus configured to processinformation received from the sensing apparatus 82 and possibly othersources in order to perform actions controlling the vehicle 10,including to steer, accelerate, and/or decelerate the vehicle 10,towards its destination on the road 19. In this embodiment, thecontroller 80 comprises an interface 166, a processing entity 168, andmemory 170, which are implemented by suitable hardware and software.

The interface 166 comprises one or more inputs and outputs (e.g., aninput/output interface) allowing the controller 80 to receive inputsignals from and send output signals to other components to which thecontroller 80 is connected (i.e., directly or indirectly connected),including the sensing apparatus 82, the powertrain 14, the steeringsystem 16, the suspension 18, and possibly other components such as theuser interface 70, a communication interface 68 configured tocommunicate over a communication network (e.g., a cellular or otherwireless network, for internet and/or other communications) and/or withone or more other vehicles that are near the vehicle 10 (i.e., forinter-vehicle communications), etc. The controller 80 may communicatewith other components of the vehicle 10 via a vehicle bus 58 (e.g., aController Area Network (CAN) bus or other suitable vehicle bus).

The processing entity 168 comprises one or more processors forperforming processing operations that implement functionality of thecontroller 80. A processor of the processing entity 168 may be ageneral-purpose processor executing program code stored in the memory170. Alternatively, a processor of the processing entity 168 may be aspecific-purpose processor comprising one or more preprogrammed hardwareor firmware elements (e.g., application-specific integrated circuits(ASICs), electrically erasable programmable read-only memories(EEPROMs), etc.) or other related elements.

The memory 170 comprises one or more memory elements for storing programcode executed by the processing entity 168 and/or data (e.g., maps,vehicle parameters, etc.) used during operation of the processing entity168. A memory element of the memory 170 may be a semiconductor medium(including, e.g., a solid-state memory), a magnetic storage medium, anoptical storage medium, and/or any other suitable type of memory. Amemory element of the memory 170 may include a read-only memory (ROM)element and/or a random-access memory (RAM) element, for example.

In some embodiments, the controller 80 may be associated with (e.g.,comprise and/or interact with) one or more other control units of thevehicle 10. For example, in some embodiments, the controller 80 maycomprise and/or interact with a powertrain control unit of thepowertrain 14, such as an engine control unit (ECU), a transmissioncontrol unit (TCU), etc.

The sensing apparatus 82 comprises sensors 90 configured to senseaspects of the environment 11 of the vehicle 10, including objects 32(e.g., people; animals; other vehicles; inanimate things;traffic-management devices such as traffic lights and traffic signs;other obstacles; lanes; free drivable areas; and/or any other tangiblestatic or dynamic objects) in that environment, sense aspects of a stateof the vehicle 10 including a position (e.g., a location, anorientation, and/or motion) of the vehicle 10, and generate dataindicative of these aspects that is provided to the controller 80 whichcan process it to determine actions to be autonomously performed by thevehicle 10 in order for the vehicle 10 to continue moving towards itsdestination.

The sensors 90 may include any suitable sensing device. For example, insome embodiments, the sensors 90 may comprise:

-   -   one or more passive sensors such as a camera 92, a sound sensor,        a light sensor, etc.;    -   one or more active sensors such as a lidar (light detection and        ranging) sensor 94 (e.g., a solid-state lidar device without        spinning mechanical components such as a microelectromechanical        system (MEMS) lidar, a flash lidar, an optical phase array        lidar, or frequency-modulated continuous wave (FMCW) lidar; or a        mechanical lidar with a rotating assembly), a radar sensor 96,        an ultrasonic sensor, etc.;    -   a location sensor 98 (e.g., based on GPS);    -   a vehicle speed sensor 97;    -   an inertial measurement unit (IMU) 95 including an        accelerometer, gyroscope, etc.; and/or    -   any other sensing device.

The vehicle 10 may be implemented in any suitable way. For example, insome embodiments, the vehicle 10, including its control system 15, maybe implemented using technology as described in https://waymo.com/tech/and https://waymo.com/safetyreport/, U.S. Pat. No. 8,818,608, or U.S.Patent Application Publication 2014/0303827, all of which areincorporated by reference herein, or using any other suitable automateddriving technology (e.g., one or more advanced driver-assistance systems(ADAS)).

With continued reference to FIG. 5, in this embodiment, the controller80 comprises a plurality of modules to autonomously drive (e.g.,accelerate, decelerate, steer, etc.) and otherwise control the vehicle10 on the road 19 towards its destination, including a perception module50 and a driving module 54. These modules may be implemented in anysuitable way in various embodiments (e.g., such as described, forinstance, in Perception, Planning, Control, and Coordination forAutonomous Vehicles by Pendleton et al., MDPI, Feb. 17, 2017, which isincorporated by reference herein, or in any known manner).

The perception module 50 is configured to provide information 210regarding perception of the environment 11 of the vehicle 10 and thestate of the vehicle 10 in real-time based on data from the sensors 90.This information 210, which will be referred to as “perceptioninformation”, conveys knowledge of the environment 11 of the vehicle 10and the vehicle's state (e.g., position, egomotion, etc.) and is used bythe driving module 54 to autonomously drive the vehicle 10.

More particularly, in this embodiment, the perception module 50 isconfigured to generate a 3D model of the environment 11 of the vehicle10 based on data from the sensors 90. This 3D model, which will bereferred to as a “3D environmental model”, comprises informationproviding a representation of the environment 11 of the vehicle 10,including objects 32 in that environment. The 3D environmental model mayinclude characteristics of these objects 32, such as their class (i.e.,type), their shape, their distance to the vehicle 10, their velocity,their position with relation to certain reference points, etc. Theperception module 50 can detect and potentially classify various objects32 in a scene of the environment 11 of the vehicle 10 using any suitableknown techniques, such as frame-based processing, segmentation,deep-learning or other machine-learning algorithms using deep neuralnetworks or other artificial neural networks, etc.

In some embodiments, as shown in FIG. 6, the perception module 50 mayinclude a sensor data fusion module 55 configured to fuse, i.e., performdata fusion to combine, integrate, and process, data from respectiveones of the sensors 90, including from the camera 92, the lidar sensor94, and possibly others such as the radar sensor 96. Such data fusionmay be implemented in any suitable way (e.g., such as described, forinstance, in U.S. Patent Application Publication 2016/0291154, which isincorporated by reference herein, or in any other known manner).

The perception module 50 is also configured to generate informationabout the position of the vehicle 10 in its environment 11 by performinglocalization of the vehicle 10 to determine its position and motion,based on data from the sensors 90, such as from the location sensor 98,the vehicle speed sensor 97, and the IMU 95. This information, whichwill be referred to as “positional information”, is indicative of theposition (e.g., the location and the orientation) of the vehicle 10and/or one or more other parameters depending on the position of thevehicle 10, such as its motion (e.g., speed, acceleration, etc.) and/orother kinematic aspects of the vehicle 10, which may be specified as itsegomotion.

Thus, in this embodiment, the perception information 210 provided by theperception module 50 includes the 3D environmental model and thepositional information for the vehicle 10 and may include otherinformation derived from the sensors 90, including the data from thesensors 90 itself.

For example, in some embodiments, the perception module 50 may beimplemented by a LeddarVision™ unit available from LeddarTech® (e.g.,https://leddartech.com/leddarvision/) or any other commerciallyavailable technology.

The driving module 54 is configured to determine how to drive (e.g.,accelerate, decelerate, and/or steer) the vehicle 10 based on theperception information 210 provided by the perception module 50,including the 3D environmental model and the positional information forthe vehicle 10, and possibly other information, and to control thevehicle 10 accordingly by sending control signals to actuators 70, suchas of the powertrain 14, the steering system 16, and/or other componentsof the vehicle 10, which control motion and/or other operational aspectsof the vehicle 10.

For instance, in this embodiment, the driving module 54 may implement aplanning module 40 to plan a safe path for the vehicle 10, such as byapplying driving policies, respecting traffic rules, making predictionsabout trajectories of the vehicle 10 and other objects in itsenvironment 11 (e.g., to avoid collisions), and/or performing othersuitable operations, and a control module 56 to generate control signalssent to the actuators 70 for autonomously moving the vehicle 10 alongthat path.

In this embodiment, the controller 80 comprises a condition detectionmodule 48 configured to determine whether one or more predefinedconditions affecting the vehicle 10 exist based on the perceptioninformation 210 provided by the perception module 50 and, if so,generate information 240 regarding existence of the predefinedcondition(s) affecting the vehicle 10. This information, which will bereferred to as “detected condition information”, can be used by thedriving module 54 to perform one or more actions concerning the vehicle10, such as adjust autonomous driving and/or other operation of thevehicle 10, convey a message regarding the vehicle 10, and/or otherwiseact to enhance performance, safety, and/or other attributes ofautonomous driving of the vehicle 10. In some cases, this may providefeedback to the driving module 54 which may otherwise be unavailableand/or may allow more rapid adjustment of autonomous driving of thevehicle 10.

A given one of the predefined conditions affecting the vehicle 10 thatcan be detected by the condition detection module 48 and indicated bythe detected condition information 240 may be environmental, i.e.,external to the vehicle 10 and resulting from the environment 11 of thevehicle 10 and generally independent from objects of interest in thescene that the driving module 54 uses to determine commands that aresent to the actuators 70. Examples of objects of interest includeadjacent vehicles and pedestrians, among others. For instance, in someembodiments, an environmental one of the predefined conditions affectingthe vehicle 10 may relate to:

-   -   the road 19, such as a shape of the road 19 (e.g., a sinuosity        or straightness of the road 19, etc.), a state of the road 19        (e.g., a slipperiness of the road 19, a roughness of the road        19, and/or other attributes of a surface of the road 19, which        may relate to a wetness or dryness of the road 19, surface        material of the road 19 (e.g., a paved road or non-paved road, a        type of pavement such as asphalt, concrete, gravel, etc.),        and/or damage (e.g., potholes, etc.) of the road 19; roadwork on        the road 19; etc.), and/or any other characteristic of the road        19;    -   an off-road area of the environment 11 of the vehicle 10, such        as one in which the vehicle 10 may have entered (e.g.,        deliberately or accidentally);    -   weather in the environment 11 of the vehicle 10, such as        precipitation (e.g., rain, sleet, snow, etc.), wind (e.g., a        speed or intensity of the wind, a direction of the wind, etc.),        temperature, fog, and/or any other characteristic of the weather        in that environment;    -   illumination in the environment 11 of the vehicle 10, such as a        type of light (e.g., sunlight, moonlight, artificial light,        outdoors, indoors such as parking or tunnel lighting, etc.), a        light intensity, and/or any other characteristic of the        illumination in that environment;    -   objects 32 in the environment 11 of the vehicle 10, such as a        density of the objects 32 (e.g., a high density indicative of        urban or other areas of relatively high traffic, a low density        indicative of suburban, rural or other areas of relatively low        traffic, etc.), distances of the objects 32 to the vehicle 10,        times for the objects 32 and the vehicle 10 to reach one another        (e.g., collide), and/or any other characteristic of the objects        32 in that environment; and/or    -   any other aspect of the environment 11 of the vehicle 10.

Alternatively, the detected condition information may be indicative ofconditions associated with the vehicle 10 and not directly associatedwith the environment 11 in which the vehicle 10 operates. Thoseconditions that can be detected by the condition detection module 48 andindicated by the detected condition information 240 are vehicular, i.e.,intrinsic to the vehicle 10 and resulting from one or more components ofthe vehicle 10, such as the powertrain 14, the steering system 16, thesuspension 18, the wheels 20, and/or any other component of the vehicle10. For example, in some embodiments, a vehicular one of the predefinedconditions affecting the vehicle 10 may relate to:

-   -   functionality of one or more components of the vehicle 10, such        as a malfunction of a component of the vehicle 10 (e.g.,        excessive vibration of a component (e.g., an engine or other        motor of the powertrain 14) of the vehicle 10; wear, damage, or        other deterioration of a component of the vehicle 10 (e.g., a        deflated or worn-out tire of a wheel 20 of the vehicle 10); a        steering anomaly (e.g., excessive freedom of movement) in the        steering system 16; a headlight not working properly; anomalous        sound generated by the powertrain 14, the steering system 16, or        the suspension 18; etc.) and/or any other dysfunction of a        component of the vehicle 10;    -   settings of one or more components of the vehicle 10, such as        power output of the powertrain 14, sensitivity (e.g., steering        wheel movement) of the steering system 16, stiffness and/or        damping of the suspension 18, and/or any other characteristic of        settings of one or more components of the vehicle 10; and/or    -   any other aspect of one or more components of the vehicle 10.

The detected condition information 240 generated by the conditiondetection module 48 and indicative of one or more predefined conditionsaffecting the vehicle 10 may thus be maintenance-related and indicativeof malfunctions or need for maintenance or adjustment.

For instance, the perception information 210 provided by the perceptionmodule 50 may be conceptually viewed as implementing two detectionstreams, namely: a main or direct one which performs detection ofobjects of interest and the output of which is used by the drivingmodule 54 to determine short-term actuator commands in order to providemotion control of the vehicle 10 into the 3D environmental model; and anancillary one which looks for predefined conditions in the environment11 that are generally independent of the objects of interest or at leastindependent of the characteristics of the objects of interest thatdetermine the short-term motion control. In some embodiments, suchdetection streams are both carried on information conveyed at least bythe lidar sensor 94 and the camera 92. In other words, informationgathered by the lidar sensor 94 and by the camera 92 is used to look forboth objects of interest for short-term motion control and also for thepredefined conditions that influence longer-term driving policy and/orvehicle maintenance.

Thus, the perception information 210 provided by the perception module50 can be further processed, other than for generating the controlsignals for motion control in the 3D environmental model, in order todetect one or more predefined conditions affecting the vehicle 10.

In this embodiment, in order to determine whether one or more predefinedconditions affecting the vehicle 10 exist, the condition detectionmodule 48 is configured to detect one or more patterns in the perceptioninformation 210 output by the perception module 50 that are indicativeof existence of one or more predefined conditions. Each of thesepatterns, which will be referred to as a “perception fingerprint”, isindicative of a predefined condition affecting the vehicle 10 such thatthe detected condition information 240 generated by the conditiondetection module 48 conveys or is otherwise based on that perceptionfingerprint.

In various examples, a given one of these perception fingerprints mayreflect a pattern in the 3D environmental model (e.g., indicative of apredefined condition related to the road 19, weather, illumination,and/or another aspect of the environment 11 of the vehicle 10), apattern in the positional information for (e.g., egomotion of) thevehicle 10 (e.g., indicative of a predefined condition related tomalfunction of the vehicle 10, such as a worn-out or deflated tire of awheel 20, a steering anomaly in the steering system 16, anomalousvibration of a motor of the powertrain 14, and/or another aspect of oneor more components of the vehicle 10), a pattern in both the 3Denvironmental model and the positional information for the vehicle 10,or a pattern in neither of the 3D environmental model and the positionalinformation for the vehicle 10 (e.g., in the data from the sensors 90).Also, a given one of these perception fingerprints may be a pattern ofdata from a combination of different ones of the sensors 90 that wouldbe undetectable by considering any of these different ones of thesensors 90 individually.

More particularly, in this embodiment, the condition detection module 48comprises a perception-fingerprint identification module 60 configuredto detect one or more perception fingerprints from the perceptioninformation 210 provided by the perception module 50 and cause thedetected condition information 240 generated by the condition detectionmodule 48 to convey or otherwise be based on these one or moreperception fingerprints.

The perception-fingerprint identification module 60 may implement anysuitable algorithm for pattern recognition to detect one or moreperception fingerprints from the perception information 210 provided bythe perception module 50. For example, in this embodiment, theperception-fingerprint identification module 60 implements artificialintelligence (AI—sometimes also referred to as machine intelligence ormachine learning), such as an artificial neural network, a supportvector machine, or any other AI unit, in software, hardware and/or acombination thereof configured to recognize perception fingerprints fromthe perception information 210 provided by the perception module 50.

More specifically, in this embodiment, shown in FIG. 7, theperception-fingerprint identification module 60 comprises an artificialneural network 64 configured to detect one or more perceptionfingerprints from the perception information 210 provided by theperception module 50. The artificial neural network 64 may be a deepneural network (e.g., convolutional, recurrent, etc.) and/or implementedusing any known kind of neural network technology.

The artificial neural network 64 is configured to learn how to detectone or more perception fingerprints from the perception information 210provided by the perception module 50. Learning by the artificial neuralnetwork 64 may be achieved using any known supervised, semi-supervised,or unsupervised technique.

In some embodiments, the artificial neural network 64 may learn during alearning mode by processing “training” data conveying information (e.g.,similar to what would be part of the perception information 210) thatone is looking for in the 3D environmental model and/or the positionalinformation for the vehicle 10, in particular data including one or moreperception fingerprints that are to be detected and thus indicative ofone or more predefined conditions affecting the vehicle 10. Forinstance, a training vehicle with sensors, a perception module, and anartificial neural network similar to the sensors 90, the perceptionmodule 50, and the artificial neural network 64 of the vehicle 10 may bedriven in situations characterized by predefined conditions of interestsuch that the perception module of the training vehicle generatestraining data that contains perception fingerprints (i.e., patterns)indicative of these predefined conditions and the artificial neuralnetwork of the training vehicle learns to identify these perceptionfingerprints by processing this training data.

For example, in some embodiments, if predefined conditions to bedetected include a rough road, a paved road, a slippery road, a sinuousroad, strong winds, heavy snow, sleet, artificial light, a worn-outtire, a deflated tire, a motor (e.g., engine) vibrating abnormally, aheadlight not working properly, a steering anomaly, anomalous sound, ora combination thereof (e.g., a rough road with strong winds, a slipperyroad with strong winds, a slippery sinuous road, a slippery sinuous roadwith strong winds, a slippery road in artificial light, a slippery roadwith worn-out tires, a rough road with deflated tires, artificial lightwith a headlight not working, etc.), or any other predefined conditionto be detected, the learning mode may involve, for each given one ofthese predefined conditions, driving the training vehicle in one or moresituations characterized by that given predefined condition (e.g., onone or more rough roads, on one or more paved roads, on one or moreslippery roads, on one or more sinuous roads, in one or more weatherevents with strong winds, in one or more weather events with heavy snow,in one or more weather events with sleet, in one or more areas withartificial light, with one or more worn-out tires, with one or moredeflated tires, with one or more steering anomalies, with one or moreanomalous motor vibrations, with one or more anomalous sounds, etc.)such that the perception module of the training vehicle generatestraining perception information that contains a perception fingerprintindicative of that given predefined condition and the artificial neuralnetwork of the training vehicle learns to identify that perceptionfingerprint.

In some embodiments, perception fingerprints detectable by theperception-fingerprint identification module 60 and predefinedconditions affecting the vehicle 10 that they are indicative of may thusbe maintained in a library or other database. In some cases, theperception-fingerprint identification module 60 may attempt to identifya perception fingerprint that has not previously been seen, in whichcases, the perception-fingerprint identification module 60 may determineif that perception fingerprint is different or anomalous with respect topreviously-encountered perception fingerprints. For instance, in aneural network implementation, a perception fingerprint may be a classof information the neural network is trained to detect by looking at thesensor data. With the embodiments in FIGS. 6 and 7, theperception-fingerprint identification module 60 may continuously outputa perception fingerprint that distinguishes the immediate environment 11in which the vehicle 10 operates among other environments the module 60is capable to identify in the perception information 210.

That perception fingerprint can be used as a further input to thedriving module 54 to condition the signals sent to the actuators 70 ofthe vehicle 10. Accordingly, the driving module 54 uses two inputs thatboth originate from the same perception information 210, in particularobject-of-interest information determining short-term motion control andenvironmental input which conditions the actual rules that determine theshort-term motion control. For example, if the environment inputindicates that the information produced by the sensors is classified ina fingerprint associated with a slippery road, that input would affectthe short-term motion control determined by the driving module 54, forinstance steering input, throttle input and brake input would bemodulated differently to account for the expected slippery surface ofthe road.

The artificial neural network 64 of the condition detection module 48may be trained to identify a perception fingerprint indicative of apredefined condition affecting the vehicle 10 from the perceptioninformation 210 provided by the perception module 50, even if thesensors 90 are not designed to directly measure the predefinedcondition. For example, in some embodiments, vibration of a motor (e.g.,an engine) of the powertrain 14 can be identified as an anomalouspattern in the positional information for (e.g., egomotion of) thevehicle 10 or a signal from the IMU 95 in the perception information210, as classification of the pattern by the artificial neural network64 indicates the source of the vibration, since the classification willbe able to separate or distinguish the vibration with its fingerprint,and natural frequency, from rough road surfaces and other phenomenaexternal to the vehicle 10 that may be at play.

With additional reference to FIG. 8, in this embodiment, the controller80 may therefore implement a process as follows.

The perception module 50 generates the perception information 210,including the 3D environmental model and the positional information forthe vehicle 10, based on the data from the sensors 90, and the drivingmodule 54 uses the perception information 210 to determine how to drive(e.g., accelerate, decelerate, and steer) the vehicle 10 and issuesignals to the actuators 70 (e.g., of the powertrain 14, the steeringsystem 16, etc.) such that the vehicle 10 is autonomously drivenaccordingly.

Meanwhile, the condition detection module 48 processes the perceptioninformation 210 provided by the perception module 50 to determinewhether it contains one or more perception fingerprints indicative ofone or more predefined conditions affecting the vehicle 10. If thecondition detection module 48 detects one or more perceptionfingerprints indicative of one or more predefined conditions affectingthe vehicle 10, the detected condition information 240 generated by thecondition detection module 48 conveys or is otherwise based on these oneor more perception fingerprints.

The driving module 54 uses the detected condition information 240, whichconveys or is otherwise based on the perception fingerprint(s)indicative of the predefined condition(s) affecting the vehicle 10, toperform one or more actions concerning the vehicle 10.

For example, in some embodiments, the driving module 54 may adjustautonomous driving and/or other operation of the vehicle 10 based on theperception fingerprint(s) detected by the condition detection module 48.For instance, in some cases, if the detected perception fingerprint(s)indicate(s) that the road 19 is rough, slippery, and/or sinuous, thereare strong winds, one or more tires of the wheels 20 are worn-out ordeflated, a motor (e.g., engine) of the powertrain 14 vibratesabnormally, there is a steering anomaly in the steering system 16, etc.,the driving module 54 may adjust the logic to determine the short-termactuator commands and autonomously drive the vehicle 10 slower (e.g.,reduce the speed of the vehicle 10 when going straight and/or turning),reduce the stiffness or increase the damping of the suspension 18, etc.Conversely, if the detected perception fingerprint(s) indicate(s) thatthe road 19 is smooth, dry, and/or straight, there is no strong wind,etc., the driving module 54 may adjust the short-term control logic toautonomously drive the vehicle 10 faster (e.g., increase the speed ofthe vehicle 10 when going straight and/or turning), increase thestiffness or decrease the damping of the suspension, etc. The drivingmodule 54 can issue signals to the actuators 70, such as of thepowertrain 14, the steering system 16, and/or the suspension 18, toadjust autonomous driving of the vehicle 10 in this way.

As another example, in some embodiments, the driving module 54 mayconvey a message regarding the vehicle 10, such as to an individual(e.g., a user of the vehicle 10) or a computing device, based on theperception fingerprint(s) detected by the condition detection module 48.The message may be indicative of malfunction or another problem with oneor more components of the vehicle 10. For instance, in some cases, thedriving module 54 may convey a notification of maintenance, repair, orother servicing to be performed on the vehicle 10 if the detectedperception fingerprint(s) indicate(s) that one or more tires of thewheels 20 are worn-out or deflated, one or more headlights are notworking, a motor (e.g., engine) of the powertrain 14 vibratesabnormally, there is a steering anomaly in the steering system 16, etc.In some embodiments, the message regarding the vehicle 10 may beconveyed to the user interface 70 of the vehicle 10. In otherembodiments, the message regarding the vehicle 10 may be conveyed to acommunication device (e.g., a smartphone or computer) that is distinct(i.e., not part of the vehicle 10, and possibly external to the vehicle10) via the communication interface 68 of the vehicle 10.

The condition detection module 48 may be configured to determine whetherone or more predefined conditions affecting the vehicle 10 exist invarious other ways in other embodiments.

For example, in some embodiments, as shown in FIG. 9, in order todetermine whether one or more predefined conditions affecting thevehicle 10 exist, the condition detection module 48 may be configured tocompare the perception information 210 provided by the perception module50 to other information 350 available to the controller 80 and distinctfrom the 3D environmental model and the positional information for(e.g., egomotion of) the vehicle 10. This information 350, which will bereferred to as “perception-independent reference information”, can beobtained from one or more sources independent from the sensors 90 usedto generate the 3D environmental model and the positional informationfor the vehicle 10. When determining that the perception information 210does not match the perception-independent reference information 350, thecondition detection module 48 determines that a predefined conditionaffecting the vehicle 10 exists and generates the detected conditioninformation 240 so that it is indicative of that predefined condition,is valid and can be used by the driving module 54 to perform one or moreactions concerning the vehicle 10, such as adjusting autonomous drivingand/or other operation of the vehicle 10 or conveying a messageregarding the vehicle 10, as discussed previously.

In some embodiments, the perception-independent reference information350 may be derived from data 67 representative of expectations relatedto the vehicle 10 (e.g., related to the environment 11 of the vehicleand/or one or more operational aspects of the vehicle 10), which may bestored in the memory 70 of the controller 80, received via thecommunication interface 68, or otherwise available to the controller 80.

As an example, in some embodiments, the perception-independent referenceinformation 350 may be derived from a map 65 (e.g., a high-definitionmap) representative of a locality of the vehicle 10, and which may bestored in the memory 70 of the controller 80, received via thecommunication interface 68, or otherwise available to the controller 80.The map 65 may provide the perception-independent reference information350, such as a kind of road surface of the road 19 that the vehicle 10should expect to encounter at a particular location (e.g., paved road,unpaved road, open country, sandy beach, etc.). The driving module 54may control the vehicle 10 based on this information provided by the map65.

By comparing the perception information 210 provided by the perceptionmodule 50 and the perception-independent reference information 350provided by the map 65, the condition detection module 48 can determinewhether the surface of the road 19 as perceived by the perception module50 (e.g., based on the 3D environmental model and/or the egomotion ofthe vehicle 10) is indeed as predicted by the map 65 and, if not,generate the detected condition information 240 so that it is indicativeof how the surface of the road 19 actually is. The driving module 54 maythen determine whether and how to adjust autonomous driving of thevehicle 10 based on the detected condition information 240. Forinstance, if the driving module 54 determines based on the detectedcondition information 240 that estimated actuator settings of theactuators 70 are improper (e.g., suboptimal or insufficient) forsmoothness of drive and safety, the driving module 54 may send signalsto the actuators 70 to adjust this accordingly.

As another example, in some embodiments, the perception-independentreference information 350 may be derived from a lighting model 34representative of expected lighting (e.g., light and shadow) around thevehicle 10, which may be stored in the memory 70 of the controller 80,received via the communication interface 68, or otherwise available tothe controller 80.

By comparing actual lighting conveyed by the perception information 210provided by the perception module 50 (e.g., based on images from thecamera 92) and the expected lighting specified by the lighting model 34of the perception-independent reference information 350, the conditiondetection module 48 can determine whether the actual lighting asperceived by the perception module 50 is indeed as predicted by thelighting model 34 and, if not, generate the detected conditioninformation 240 so that it is indicative of the actual lighting. Thedriving module 54 may then determine whether and how to adjustautonomous driving of the vehicle 10 based on the detected conditioninformation 240. For instance, if the driving module 54 determines basedon the detected condition information 240 that settings of the actuators70 are improper (e.g., suboptimal or insufficient) for smoothness ofdrive and safety, the driving module 54 may send signals to theactuators 70 to adjust this accordingly. Alternatively or additionally,the driving module 54 may send a message indicating that maintenance orother servicing is to be performed on the vehicle 10.

In some embodiments, as shown in FIG. 10, the perception-independentreference information 350 may be derived from the powertrain 14, thesteering system 16, the suspension 18, and/or any other componentcontrolling motion of the vehicle 10. For example, in some embodiments,the perception-independent reference information 350 may be indicativeof steering movement of steered ones of the wheels 20 effected by thesteering system 16 as reported on the vehicle bus 58 (e.g., CAN bus),while the egomotion of the vehicle 10 included in the perceptioninformation 210 provided by the perception module 50 can be used toestimate perceived (e.g., past) steering movement of the steered ones ofthe wheels 20.

By comparing the perceived steering wheel movement with the reportedsteering wheel movement, the condition detection module 48 can determinewhether the steering wheel movement as perceived by the perceptionmodule 50 indeed corresponds to the steering wheel movement as reportedon the vehicle bus 58 and, if not, generate the detected conditioninformation 240 so that it is indicative of what the steering wheelmovement actually is. The driving module 54 may then determine whetherand how to adjust autonomous driving of the vehicle 10 based on thedetected condition information 240. For instance, if the driving module54 determines based on the detected condition information 240 thatestimated actuator settings of respective ones of the actuators 70 inthe steering system 16 are improper (e.g., suboptimal or insufficient)for steerability, the driving module 54 may send signals to theseactuators 70 to adjust this accordingly. Alternatively, or additionally,the driving module 54 may send a message indicating that maintenance orother servicing is to be performed on the vehicle 10.

As another example, in some embodiments, in order to determine whetherone or more predefined conditions affecting the vehicle 10 exist, thecondition detection module 48 may be configured to monitor temporalvariation (i.e., variation in time) of the perception information 210provided by the perception module 50. For instance, the conditiondetection module 48 may monitor temporal variation of parameters thatdepend on the 3D environmental model and, when observing that one ormore of these parameters of the 3D environmental model vary in time in aprescribed way deemed to be indicative of a predefined conditionaffecting the vehicle 10, the condition detection module 48 generatesthe detected condition information 240 so that it is indicative of thatpredefined condition and can be used by the driving module 54 to performone or more actions concerning the vehicle 10, such as adjustingautonomous driving and/or other operation of the vehicle 10 or conveyinga message regarding the vehicle 10, as discussed previously.

For instance, in some embodiments, the condition detection module 48 maymonitor a time-dependent statistical behavior of the 3D environmentalmodel. For example, a distribution of “distance to obstacle” or “time tocollision” for objects 32 in the environment 11 of the vehicle 10 may bemonitored. Desirable behavior within a given driving scenario might bethat changes to that distribution are slow and smooth (e.g., below athreshold rate). Control of the vehicle 10 by the driving module 54 isdetermined by driving policy, and tracking statistics of theenvironmental model distribution may help to evaluate different policiesand adjust between them.

In another variant, a perception fingerprint may be used solely forvehicle maintenance purposes, without impact on motion control. In suchinstance, the perception-fingerprint identification module 60 may, inaddition to camera and lidar data, receive an input from drivetrainsensors configured to detect specific malfunctions or drivetrainconditions. In this instance, the condition detection module 48 wouldprovide a higher level of intelligence in fault detection and trigger amaintenance message when the actual impact of a fault condition,reported by a drivetrain sensor, is observed in the 3D environmentalmodel.

While in embodiments considered above the vehicle 10 travels on land,the vehicle 10 may travel other than on land in other embodiments. Forexample, in other embodiments, the vehicle 10 may fly (e.g., a deliverydrone or other unmanned aerial vehicle, a flying car or other personalair vehicle, etc.) or travel on water (e.g., a water taxi or otherboat), such that “driving” generally means operating, controlling, anddirecting a course of the vehicle 10.

Certain additional elements that may be needed for operation of someembodiments have not been described or illustrated as they are assumedto be within a purview of those of ordinary skill. Moreover, certainembodiments may be free of, may lack and/or may function without anyelement that is not specifically disclosed herein.

Any feature of any embodiment discussed herein may be combined with anyfeature of any other embodiment discussed herein in some examples ofimplementation.

In case of any discrepancy, inconsistency, or other difference betweenterms used herein and terms used in any document incorporated byreference herein, meanings of the terms used herein are to prevail andbe used.

Although various embodiments and examples have been presented, this wasfor purposes of describing, but is not limiting. Various modificationsand enhancements will become apparent to those of ordinary skill.

1. A system for autonomous driving of a vehicle, the system comprising:an interface configured to receive data from sensors of the vehicle thatinclude a camera and a lidar sensor; and a processing entity comprisingat least one processor and configured to: provide perception informationregarding perception of an environment of the vehicle and a state of thevehicle based on the data from the sensors, the perception informationcomprising a 3D model of the environment of the vehicle and informationabout a position of the vehicle; generate control signals forautonomously driving the vehicle based on the 3D model of theenvironment of the vehicle and the information about the position of thevehicle; and process the perception information, other than forgenerating the control signals for autonomously driving the vehiclebased on the 3D model of the environment of the vehicle and theinformation about the position of the vehicle, to determine whether apredefined condition affecting the vehicle exists and, if so, perform anaction concerning the vehicle based on the predefined condition.
 2. Thesystem of claim 1, wherein, to determine that the predefined conditionaffecting the vehicle exists, the processing entity is configured todetect a pattern in the perception information indicative of thepredefined condition.
 3. The system of claim 2, wherein the processingentity comprises an artificial neural network trained to detect thepattern in the perception information indicative of the predefinedcondition.
 4. The system of any one of claims 2 and 3, wherein thepattern in the perception information indicative of the predefinedcondition is in the 3D model of the environment of the vehicle.
 5. Thesystem of claim any one of claims 2 and 3, wherein the pattern in theperception information indicative of the predefined condition is in theinformation about the position of the vehicle.
 6. The system of claimany one of claims 2 and 3, wherein the pattern in the perceptioninformation indicative of the predefined condition is in both the 3Dmodel of the environment of the vehicle and the information about theposition of the vehicle.
 7. The system of claim any one of claims 2 and3, wherein the pattern in the perception information indicative of thepredefined condition is in neither of the 3D model of the environment ofthe vehicle and the information about the position of the vehicle. 8.The system of any one of claims 2 and 3, wherein the pattern in theperception information indicative of the predefined condition arisesfrom a combination of different ones of the sensors and is undetectablefrom any of the different ones of the sensors individually.
 9. Thesystem of claim 1, wherein, to determine that the predefined conditionaffecting the vehicle exists, the processing entity is configured tocompare the perception information to reference information.
 10. Thesystem of claim 9, wherein the reference information is derived from amap representative of a locality of the vehicle.
 11. The system of claim10, wherein: the vehicle travels on a road; and the referenceinformation is indicative of a state of the road according to the map.12. The system of claim 9, wherein the reference information is derivedfrom a component controlling motion of the vehicle.
 13. The system ofclaim 12, wherein the reference information is derived from a vehiclebus connected to the component controlling motion of the vehicle. 14.The system of any one of claims 12 and 13, wherein the componentcontrolling motion of the vehicle is a powertrain of the vehicle. 15.The system of any one of claims 12 and 13, wherein the componentcontrolling motion of the vehicle is a steering system of the vehicle.16. The system of claim 1, wherein, to determine that the predefinedcondition affecting the vehicle exists, the processing entity isconfigured to monitor temporal variation of the perception information.17. The system of any one of claims 1 to 16, wherein the processingentity is configured to perform data fusion on the data from respectiveones of the sensors, including the camera and the lidar sensor, toprovide the perception information.
 18. The system of any one of claims1 to 17, wherein the predefined condition affecting the vehicle isexternal to the vehicle and results from the environment of the vehicle.19. The system of claim 18, wherein the predefined condition affectingthe vehicle relates to a road on which the vehicle travels.
 20. Thesystem of claim 19, wherein the predefined condition affecting thevehicle relates to a state of the road.
 21. The system of claim 20,wherein the predefined condition affecting the vehicle relates to aslipperiness of the road.
 22. The system of any one of claims 20 and 21,wherein the predefined condition affecting the vehicle relates to aroughness of the road.
 23. The system of any one of claims 20 to 22,wherein the predefined condition affecting the vehicle relates tosurface material of the road.
 24. The system of any one of claims 19 to23, wherein the predefined condition affecting the vehicle relates to ashape of the road.
 25. The system of claim 24, wherein the predefinedcondition affecting the vehicle relates to a sinuosity of the road. 26.The system of claim 18, wherein the predefined condition affecting thevehicle relates to weather in the environment of the vehicle.
 27. Thesystem of claim 26, wherein the predefined condition affecting thevehicle relates to precipitation in the environment of the vehicle. 28.The system of any one of claims 26 and 27, wherein the predefinedcondition affecting the vehicle relates to wind in the environment ofthe vehicle.
 29. The system of claim 18, wherein the predefinedcondition affecting the vehicle relates to illumination in theenvironment of the vehicle.
 30. The system of claim 18, wherein thepredefined condition affecting the vehicle relates to a density ofobjects in the environment of the vehicle.
 31. The system of any one ofclaims 1 to 17, wherein the predefined condition affecting the vehicleis intrinsic to the vehicle and results from a component of the vehicle.32. The system of claim 31, wherein the predefined condition affectingthe vehicle relates to functionality of the component of the vehicle.33. The system of claim 32, wherein the predefined condition affectingthe vehicle relates to malfunction of the component of the vehicle. 34.The system of claim 33, wherein the predefined condition affecting thevehicle relates to deterioration of the component of the vehicle. 35.The system of claim 34, wherein: the component of the vehicle is a tireof a wheel of the vehicle; and the predefined condition affecting thevehicle relates to wear of the tire.
 36. The system of claim 34,wherein: the component of the vehicle is a tire of a wheel of thevehicle; and the predefined condition affecting the vehicle relates todeflation of the tire.
 37. The system of claim 31, wherein thepredefined condition affecting the vehicle relates to vibration of thecomponent of the vehicle.
 38. The system of claim 31, wherein: thecomponent of the vehicle is a steering system of the vehicle; and thepredefined condition affecting the vehicle relates to a steering anomalyof the steering system.
 39. The system of claim 31, wherein: thecomponent of the vehicle is a headlight of the vehicle; and thepredefined condition affecting the vehicle relates to the headlight ofthe vehicle not working properly.
 40. The system of claim 31, whereinthe predefined condition affecting the vehicle relates to settings ofthe component of the vehicle.
 41. The system of any one of claims 1 to40, wherein the action concerning the vehicle comprises an adjustment ofautonomous driving of the vehicle based on the predefined condition. 42.The system of claim 41, wherein the adjustment of autonomous driving ofthe vehicle comprises a variation of a speed of the vehicle.
 43. Thesystem of any one of claims 1 to 40, wherein the action concerning thevehicle comprises generation of a signal directed to a component of thevehicle based on the predefined condition.
 44. The system of claim 43,wherein the component of the vehicle is a powertrain of the vehicle. 45.The system of claim 43, wherein the component of the vehicle is asteering system of the vehicle.
 46. The system of claim 43, wherein thecomponent of the vehicle is a suspension of the vehicle.
 47. The systemof any one of claims 1 to 40, wherein the action concerning the vehiclecomprises conveyance of a message regarding the vehicle.
 48. The systemof claim 47, wherein the message regarding the vehicle is conveyed to auser interface of the vehicle.
 49. The system of claim 47, wherein themessage regarding the vehicle is conveyed to a communication devicedistinct from the vehicle.
 50. The system of any one of claims 47 to 49,wherein the message regarding the vehicle is indicative of malfunctionof a component of the vehicle.
 51. The system of any one of claims 1 to50, wherein: the predefined condition is one of a plurality ofpredefined conditions affecting the vehicle; and the processing entityis configured to process the perception information to determine whetherany one of the predefined conditions affecting the vehicle exists and,if so, perform an action concerning the vehicle based on each of thepredefined conditions determined to exist.
 52. A system for autonomousdriving of a vehicle, the system comprising: an interface configured toreceive data from sensors of the vehicle that include a camera and alidar sensor; and a processing entity comprising at least one processorand configured to: provide perception information regarding perceptionof an environment of the vehicle and a state of the vehicle based on thedata from the sensors, the perception information comprising a 3D modelof the environment of the vehicle and information about a position ofthe vehicle; generate control signals for autonomously driving thevehicle based on the 3D model of the environment of the vehicle and theinformation about the position of the vehicle; and process theperception information to detect a pattern in the perception informationindicative of a predefined condition affecting the vehicle. 53.Non-transitory computer-readable media comprising instructionsexecutable by a processing apparatus for autonomous driving of avehicle, wherein the instructions, when executed by the processingapparatus, cause the processing apparatus to: receive data from sensorsof the vehicle that include a camera and a lidar sensor; provideperception information regarding perception of an environment of thevehicle and a state of the vehicle based on the data from the sensors,the perception information comprising a 3D model of the environment ofthe vehicle and information about a position of the vehicle; generatecontrol signals for autonomously driving the vehicle based on the 3Dmodel of the environment of the vehicle and the information about theposition of the vehicle; and process the perception information, otherthan for generating the control signals for autonomously driving thevehicle based on the 3D model of the environment of the vehicle and theinformation about the position of the vehicle, to determine whether apredefined condition affecting the vehicle exists and, if so, perform anaction concerning the vehicle based on the predefined condition. 54.Non-transitory computer-readable media comprising instructionsexecutable by a processing apparatus for autonomous driving of avehicle, wherein the instructions, when executed by the processingapparatus, cause the processing apparatus to: receive data from sensorsof the vehicle that include a camera and a lidar sensor; provideperception information regarding perception of an environment of thevehicle and a state of the vehicle based on the data from the sensors,the perception information comprising a 3D model of the environment ofthe vehicle and information about a position of the vehicle; generatecontrol signals for autonomously driving the vehicle based on the 3Dmodel of the environment of the vehicle and the information about theposition of the vehicle; and process the perception information todetect a pattern in the perception information indicative of apredefined condition affecting the vehicle.
 55. A method for autonomousdriving of a vehicle, the method comprising: receiving data from sensorsof the vehicle that include a camera and a lidar sensor; providingperception information regarding perception of an environment of thevehicle and a state of the vehicle based on the data from the sensors,the perception information comprising a 3D model of the environment ofthe vehicle and information about a position of the vehicle; generatingcontrol signals for autonomously driving the vehicle based on the 3Dmodel of the environment of the vehicle and the information about theposition of the vehicle; and processing the perception information,other than for generating the control signals for autonomously drivingthe vehicle based on the 3D model of the environment of the vehicle andthe information about the position of the vehicle, to determine whethera predefined condition affecting the vehicle exists and, if so, performan action concerning the vehicle based on the predefined condition. 56.A method for autonomous driving of a vehicle, the method comprising:receiving data from sensors of the vehicle that include a camera and alidar sensor; providing perception information regarding perception ofan environment of the vehicle and a state of the vehicle based on thedata from the sensors, the perception information comprising a 3D modelof the environment of the vehicle and information about a position ofthe vehicle; generating control signals for autonomously driving thevehicle based on the 3D model of the environment of the vehicle and theinformation about the position of the vehicle; and processing theperception information to detect a pattern in the perception informationindicative of a predefined condition affecting the vehicle.